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This study examined the extent to which treatment integrity of 4 special education teachers was 
affected by goal setting, performance feedback regarding student or teacher performance, and 
a meeting cancellation contingency. Teachers were trained to implement function-based 
treatment packages to address student problem behavior. In one condition, teachers set a goal for 
student behavior and received daily written feedback about student performance. In a second 
condition, teachers received daily written feedback about student performance as well as their 
own accuracy in implementing the intervention and would be able to avoid meeting with 
a consultant to practice missed steps by implementing the intervention with 100% integrity. 
This latter package increased treatment integrity the most above baseline levels. Higher levels of 
treatment integrity were significantly correlated with lower levels of student problem behavior for 
3 of the 4 teacher-student dyads. Three of the 4 teachers also rated both feedback procedures as 
highly acceptable. Implications for increasing and maintaining treatment integrity by teachers via 
a consultation model are discussed. 
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Recent demands for high levels of account- 
ability in our education system (e.g., Individuals 
with Disabilities Education Improvement Act 
of 2004, No Child Left Behind Act of 2001) 
have placed increased pressure on educators to 
use evidence-based practices (Vaughn & Fuchs, 
2003; Walker, 2004). The use of evidence-based 
practices is particularly important during 
school consultation in which a psychologist 
(consultant) works cooperatively with a teacher 
to design, implement, and evaluate an in- 
tervention plan for a student (Erchul & 
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Martens, 2002). During the consultation pro- 
cess, responsibility for plan implementation 
rests primarily with teachers (Gutkin & Curtis, 
1999). Moreover, most intervention plans 
require teachers to acquire new instructional 
and behavior-management skills and to in- 
corporate these skills into their teaching 
repertoire (Martens & DiGennaro, in press). 
A critical aspect of effective consultation, 
therefore, is to ensure that teachers have 
acquired the skills needed for plan implemen- 
tation and that teacher behavior has changed 
(Erchul & Martens). 

The extent to which teachers implement 
school-based interventions consistently and 
accurately has been referred to as treatment 
integrity (Gresham, 1989; Noell et ah, 2000). 
Assessment of treatment integrity is typically 
accomplished by directly observing teachers 
during plan implementation and calculating 
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the percentage of treatment steps implemented 
correctly (e.g., DiGennaro, Martens, & McIn- 
tyre, 2005; Wickstrom, Jones, LaFleur, & Witt, 
1998). A series of investigations by Noell and 
his colleagues examined teachers’ treatment 
integrity after initial training and following 
implementation of performance feedback (Mor- 
tenson & Witt, 1998; Noell, Duhon, Gatti, & 
Connell, 2002; Noell, Witt, Gilbertson, Ranier, 
& Freeland, 1997; Witt, Noell, LaFleur, & 
Mortenson, 1997). For example, Noell et al. 
(2002) examined the effectiveness of data review 
and performance feedback on the integrity with 
which 4 general education teachers implemen- 
ted intervention plans. During data review, the 
consultant met with the teachers for 3 to 5 min 
to review a monitoring form, missed steps of the 
intervention, plan implementation, student 
functioning, and to solve problems for the 
following session. During performance feed- 
back, the consultant no longer reviewed the 
monitoring form. Instead, two graphs depicting 
student progress and teacher integrity were 
shared with the teacher. These data reviews 
increased treatment integrity for 1 teacher, 
slightly improved integrity for 2 teachers, and 
had no effect on a 4th teacher. Application of 
performance feedback resulted in high levels of 
treatment integrity for all teachers. 

Previous studies have also shown that the 
integrity of plan implementation by teachers 
may be affected by reinforcement contingencies 
(Gillat & Sulzer-Azaroff, 1994; Noell et al., 
1997, 2000) in much the same way that student 
behavior is subject to contingencies of reinforce- 
ment (see also Lentz & Daly, 1996; Martens & 
Witt, 1988; Tharp & Wetzel, 1969). For 
example, Ward, Johnson, and Konukman 
(1998) established a negative reinforcement 
contingency by asking preservice teachers to 
rehearse 10 times those teaching behaviors that 
were implemented incorrectly during a physical 
education practicum course. Results revealed 
that all teachers reached 100% correct steps 
under the directed rehearsal condition. The 


authors speculated that an effective avoidance 
contingency was established, under which 
teachers were motivated to exhibit appropriate 
teaching behaviors to avoid meeting with the 
consultant. 

DiGennaro et al. (2005) continued the 
analysis of strategies for improving treatment 
integrity by evaluating the extent to which daily 
performance feedback, practice, and the oppor- 
tunity to avoid meetings with the consultant 
influenced the treatment integrity of 4 teachers 
and reduced students’ problem behaviors. 
Teachers were instructed to use directed re- 
hearsal (repeating missed intervention steps 
three times) during a brief meeting with 
a consultant, thus establishing a negative re- 
inforcement contingency for correct implemen- 
tation. Results of the study showed that 
integrity increased to 100% for all teachers 
following implementation of performance feed- 
back. Integrity was maintained at high levels 
when the package was faded to once a week and 
then once every 2 weeks. Moderate decreases in 
problem behavior were also observed for all 4 
students. 

Researchers have also examined the use of 
goal setting in combination with performance 
feedback (Martens, Hiralall, & Bradley, 1997). 
Martens et al. asked a teacher to identify 
appropriate alternative behaviors to praise in 2 
students who exhibited problem behaviors. The 
teacher was then asked to set a goal for herself 
regarding the number of praise statements she 
would provide for these behaviors within daily 
30-min sessions. Following each session, the 
teacher was provided with written feedback as 
to whether she met her goal. Results revealed 
increases in appropriate student behavior and 
the number of praise statements provided by the 
teacher. In addition, the intervention was rated 
as highly acceptable by the teacher. 

Taken together, these findings suggest that 
teachers’ treatment implementation is sensitive 
to contingencies of reinforcement, that teachers 
require ongoing support from consultants to 
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implement intervention plans consistently and 
accurately, and that teachers can maintain high 
levels of treatment integrity following pro- 
gressive thinning of performance feedback and 
reinforcement schedules. Furthermore, collater- 
al effects on student performance as a result of 
intervention implementation have been re- 
ported (DiGennaro et ah, 2005). 

Despite these findings, previous research has 
not examined whether daily written feedback 
about student progress toward an agreed-upon 
goal is sufficient to maintain high rates of 
accurate plan implementation by teachers. 
Support provided in this manner requires 
minimal time investment on the part of both 
teacher and consultant compared to previously 
examined treatment packages. One goal of this 
study was to examine teachers’ treatment 
integrity after setting a goal for student behavior 
and then receiving feedback about student 
progress toward that goal. In this condition, 
contingencies for treatment integrity were not 
programmed or communicated explicitly. Thus, 
a unique contribution of this study was 
examination of the extent to which observing 
students’ progress toward a stated goal would 
affect treatment implementation by teachers. 

A second goal of this study was to replicate 
the findings of DiGennaro et al. (2005) and to 
extend these to special education teachers. 
Specifically, we compared the effects of goal 
setting and performance feedback about student 
behavior to a condition in which teachers 
received daily written feedback about their 
own implementation accuracy. In this latter 
condition, teachers were also able to avoid 
meeting with a consultant to practice missed 
steps by implementing the intervention with 
1 00% integrity. A third goal of this study was to 
determine if high levels of treatment integrity 
would be maintained when the schedule of 
feedback was progressively thinned from daily 
to once every 2 weeks, thus further replicating 
the findings of Noell et al. (2000) and 
DiGennaro et al. Finally, a fourth aim of this 


study was to replicate previous findings that 
suggested a relation between teacher treatment 
integrity and student behavior (e.g., DiGennaro 
et al.). 

METHOD 

Participants and Setting 

Four special education teachers, employed in 
a setting that provides educational and residen- 
tial services to students with brain injuries, were 
recruited by the first author. Each teacher 
sought assistance to address the off-task behav- 
ior, work refusal, and disruptive verbalizations 
of particular students. These 4 students also 
served as participants. Teachers were informed 
that the study investigated different ways of 
helping teachers implement school-based inter- 
ventions, and that trained observers would visit 
the classroom daily at specified times. 

Dyad A. Teacher A, a Haitian Creole male 
teacher of 1 year, taught a special education 
class comprised of 6 students. He had a BA and 
was certified in elementary education in two 
states. Student A, a 13-year-old Caucasian boy, 
had attended the school for just over 3 years. He 
had suffered a right hemispheric stroke followed 
by hydrocephalus for which a V-P shunt was 
successfully placed when he was 1 year old. He 
also experienced left hemiplegia, seizure disor- 
der, anxiety, attentional difficulties, and mod- 
erate mental retardation. His prescription 
medications included Abilify® and Tenex®. 
The teacher indicated that the problem behav- 
iors of most concern were the student’s refusal 
to complete assigned work and lack of active 
involvement in classroom group activities. 
Therefore, the identified target behavior for 
Student A was off-task behavior, defined as any 
motor behaviors that lasted a minimum of 3 s 
or verbalizations that were not permitted or 
were unrelated to the assigned academic task. 
Before an intervention was introduced (i.e., 
during baseline), Student A successfully avoided 
and escaped small-group instruction by engag- 
ing in a variety of off-task behaviors (e.g., placed 
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Table 1 

Intervention Components for Student A 


1 . Allow the student to select a reward icon from back of chart (reward that allows him to leave work area) and move the icon to the 
choice box. 

2. Tell student, “After you get five stickers for participating in group you can earn a break.” 

3. After 3 min the teacher should present the student with a question or instruction. Remind the student that he can earn a sticker if he 

tries to answer or follows the direction. 

4. Student will earn one sticker paired with verbal praise if he attempts to respond even if he responds with an incorrect answer. 

3. After 6 min the teacher should present the student with a question or instruction. Remind the student that he can earn a sticker if he 
tries to answer or follows the direction. 

6. Student will earn one sticker paired with verbal praise if he attempts to respond even if he responds with an incorrect answer. 

7. After 9 min the teacher should present the student with a question or instruction. Remind the student that he can earn a sticker if he 

tries to answer or follows the direction. 

8. Student will earn one sticker paired with verbal praise if he attempts to respond even if he responds with an incorrect answer. 

9. After 12 min the teacher should present the student with a question or instruction. Remind the student that he can earn a sticker if he 

tries to answer or follows the direction. 

10. Student will earn one sticker paired with verbal praise if he attempts to respond even if he responds with an incorrect answer. 

1 1 . After 1 3 min the teacher should present the student with a question or instruction. Remind the student that he can earn a sticker if 
he tries to answer or follows the direction. 

12. Student will earn one sticker paired with verbal praise if he attempts to respond even if he responds with an incorrect answer. 

13. After the student earns five stickers he should receive his reward of a break. 


his head on the desk, turned his body from 
the lesson, responded “who cares?”). In addi- 
tion, consistent consequences for staying on 
task and completing work were not observed. 
As a result, we implemented an intervention 
that allowed Student A to earn token re- 
inforcement (stickers) about once every 3 min 
and then exchange these tokens to leave group 
instruction (e.g., take a walk) when five had 
been acquired (see Table 1 for an intervention 
example). 

Dyad B. Teacher B, an Asian female teacher 
of 3 years, was close to completing her MS 
degree in applied behavior analysis. She taught 
a class comprised of 6 students. Student B, 
a Caucasian boy of 14 years, suffered from 
cavernous hemangioma characterized by lesions 
in the vascular system and brain, attention 
deficit hyperactivity disorder, and pervasive 
developmental disorder (not otherwise speci- 
fied). He had been attending the program for 
just over 6 years. He had been prescribed 
Flonase®, Risperdal®, DDAVP®, Zantac®, 
clonidine, and Lactaid®. He had been referred 
for a variety of off-task behaviors including 
work refusal, disruption, sleeping, and teasing 
others. Off-task behavior was defined as any 
motor behaviors lasting a minimum of 3 s or 


verbalizations that were not permitted or were 
unrelated to the assigned academic task. Because 
the student was able to escape the lesson by 
engaging in these behaviors during baseline, 
the teacher was asked to implement an in- 
tervention plan similar to that described for 
Student A (on-task behavior was rewarded with 
escape from task demands and access to an 
alternative activity within a token reinforcement 
program). 

Dyad C. Teacher C, a Caucasian man, had 
been a special education teacher for 4.5 years; 
he had an MA degree in education and 
preliminary certification in special education. 
He taught a class of 5 students. Student C was 
a 21 -year-old Caucasian man who had been 
diagnosed with tuberous sclerosis, seizure dis- 
order, and vascular tumors on the face and 
body. He had been enrolled in the school for 
just under 4 years. He had been prescribed 
Risperdal®, valproic acid, Dilantin®, amoxicil- 
lin, Mederma® gel, benzoyl peroxide, and 
ketoconazole. Teacher C referred his student 
due to his refusal to participate in classroom 
group activities, defined as any verbal (e.g., 
swearing) or gestural (e.g., flopping on the 
floor, positioning his body 90° in his chair away 
from the group) refusal to a request made by the 
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teacher. During baseline, Student C successfully 
escaped group instruction by engaging in these 
behaviors. An intervention was introduced that 
allowed him to leave the area of group 
instruction after 3 min without work refusals 
and to relax in a beanbag chair for 2 min. 

Dyad D. Teacher D, a Caucasian woman, 
had been a teacher in a special education class of 
6 students for 1 year. She had an MA degree in 
elementary education and was certified in 
preschool and elementary education. Student 
D was a 9-year-old Caucasian girl who had 
attended the school for just over 3 years. She 
had been diagnosed with subdural and sub- 
arachnoid hemorrhages and seizures secondary 
to shaken baby syndrome. She had also been 
diagnosed with traumatic brain injury, micro- 
cephaly, lead poisoning, and legal blindness. 
She presented with features associated with 
a diagnosis of autism and commonly engaged in 
self-injurious head banging. She had been 
prescribed trazodone, Risperdal®, Topamax®, 
Depakote®, and Ambien®, some of which were 
adjusted during the study. The problem 
behavior of most concern was off-task behavior, 
defined as any motor behaviors that lasted 
a minimum of 3 s or verbalizations that were 
not permitted or were unrelated to the assigned 
academic task. Common behaviors included 
walking around the room, flopping on the 
floor, and looking through books during times 
when this was not allowed. During baseline, 
consistent consequences for on-task behavior 
were not observed. In addition, Student D was 
allowed access to books as she desired and was 
observed to walk around the room during 
instructional time. Thus, an intervention was 
designed that provided teacher attention every 
10 to 15 s for being on task and 1-min access to 
a preferred item for every 1 min of on-task 
behavior. 

The first author served as the consultant for 
all four dyads. Teacher interviews, training, 
plan implementation, and performance feed- 
back occurred in the teachers’ classrooms. 


Classrooms were similar in size and contained 
a teacher’s desk, a computer, bulletin boards, 
a small-group teaching station, and individual 
student workstations consisting of a desk or 
small table. During observations, Student A was 
seated at his desk behind the small group with 
approximately three adults present. Students B, 
C, and D were seated in their classroom groups 
with three adults, four adults, and five adults 
present, respectively. During small-group in- 
struction, students were exposed to grade- 
appropriate general education curriculum (e.g., 
history, science, and reading) that was modified 
given the developmental level of the students. 

Dependent Measures 

Treatment integrity. The primary dependent 
measure was the integrity with which teachers 
implemented the agreed-upon plans. Treatment 
integrity was assessed through daily 15-min 
direct observations of the teacher by the first 
and third authors and was calculated by 
dividing the number of 13 treatment steps 
implemented correctly during the observation 
period by the total number of treatment steps, 
multiplied by 100% (see Table 1 for an 
example of the 13 steps for Student A). 
Teachers were observed at the same time each 
day. A treatment step was considered correct if 
it (a) was implemented as written or (b) was 
implemented within 30 s of the specified time, 
if indicated. 

Treatment effectiveness. To assess the effec- 
tiveness of the intervention, the first and third 
authors also collected data on student problem 
behavior. This observation was conducted 
during the treatment integrity observation 
(e.g., a 15-min observation conducted at the 
same time each day). Partial-interval 10-s 
recording was used to score occurrences of 
target behavior across all phases. The percentage 
of intervals in which students exhibited target 
behavior was calculated by dividing the number 
of intervals during which target behavior 
occurred by the total number of intervals 
multiplied by 100%. 
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Interobserver Agreement and Procedural Fidelity 

A second observer simultaneously collected 
data on teacher treatment integrity during 30%, 
43%, 37%, and 47% of sessions for Teachers A, 
B, C, and D, respectively, to assess interobserver 
agreement. The observers were in agreement 
when they independently scored the teacher’s 
implementation of a treatment step similarly 
(i.e., as correct or incorrect). Interobserver 
agreement was calculated as the number of 
instances of agreement divided by agreements 
plus disagreements multiplied by 100%. Mean 
percentage of agreement for Teacher A was 
97% (range, 85% to 100%), for Teacher B was 
97% (range, 92% to 100%), for Teacher C was 
100%, and for Teacher D was 94% (range, 
77% to 100%). A second observer also collected 
data on student target behavior during 30%, 
40%, 49%, and 43% of sessions for Students A, 
B, C, and D, respectively. Interobserver agree- 
ment was scored on an interval-by-interval basis 
and was calculated as the number of agreements 
divided by agreements plus disagreements 
multiplied by 100%. Mean percentage agree- 
ment for Student A was 85% (range, 77% to 
92%), for Student B was 91% (range, 78% to 
100%), for Student C was 98% (range, 95% to 
100%), and for Student D was 91% (range, 
75% to 100%). 

Treatment acceptability. Teachers completed 
the Intervention Rating Profile- 15 (IRP-15) 
(Martens, Witt, Elliot, & Darveaux, 1985) to 
assess their judgments about the intervention 
used with their students. The IRP-15 contains 
15 items rated on a six-point Likert-type scale 
(1 = strongly disagree to 6 = strongly agree), with 
the total score used as a global index of 
intervention acceptability. This instrument has 
been shown to have high internal consistency 
(Cronbach’s alpha = .98). Previous factor 
analyses revealed that all items loaded onto 
a single factor, termed general acceptability 
(Martens et al.). Teachers were also asked to 
complete two additional acceptability question- 
naires modified from the IRP-15. These 


questionnaires were created for the purposes of 
this study to evaluate the teachers’ judgments 
about the specific interventions. The number of 
questions was a reflection of the number of 
interventions included within a condition. In 
one condition, a package of three interventions 
was used with teachers. As a result, the 
acceptability questionnaire had a higher number 
of items than the questionnaire inquiring about 
acceptability of a condition with only two 
interventions. A copy of the adapted version of 
this scale is available from the first author. 

Experimental Design and Procedure 

A multiple baseline design across the four 
dyads was used to evaluate the effects of goal 
setting, child and teacher performance feedback, 
directed rehearsal, and meeting cancellation on 
teachers’ treatment integrity and student behav- 
ior. The treatment integrity analysis included 
six phases: (a) pretraining baseline, (b) training, 
(c) implementation baseline, (d) goal setting 
and student performance feedback, (e) teacher 
performance feedback and directed rehearsal 
with meeting cancellation, and (1) fading. The 
first three phases occurred in the order specified 
above for all teachers. Goal setting and student 
performance feedback preceded teacher perfor- 
mance feedback and directed rehearsal with 
meeting cancellation for Teachers A and B, 
whereas teacher performance feedback and 
directed rehearsal with meeting cancellation 
occurred prior to goal setting and student 
performance feedback for Teachers C and D. 
A brief reversal to teacher performance feedback 
and directed rehearsal with meeting cancellation 
occurred for these individuals as well. Thus, 
Teachers A and B were exposed to the 

staggering of phases in the order of ABCDE. 
Teachers C and D were exposed to the 

staggering of phases in the order of ABCEDE. 
The final phase consisted of fading for all 
teachers. 

Pretraining baseline. During this phase, 
teachers were asked to teach their classes and 
respond to any behavioral difficulty in accor- 
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dance with the behavior plan already in place 
for each student. Existing intervention plans did 
not address target behaviors identified for this 
study for Students A, B, and D. During 
pretraining baseline, teachers had not yet been 
trained on the intervention procedures. The 
next phase was introduced after a minimum of 
four classroom observations had been con- 
ducted and student behavior was generally 
stable. 

Training. Initial training in the various steps 
of the intervention occurred in the teachers’ 
classrooms and was comprised of didactic 
instruction, modeling, coaching, and immediate 
corrective feedback. The intervention consisted 
of differential negative reinforcement of alter- 
native behaviors (DNRA) and differential re- 
inforcement of alternative behaviors (DRA) and 
was based on the results of a functional 
assessment for each student. Following both 
indirect and descriptive functional assessments, 
the consultant met with each teacher individ- 
ually, reviewed the function-based intervention 
plan, modeled the intervention steps, answered 
questions, and obtained agreement to imple- 
ment the plan. The following day, the consul- 
tant began training while the teacher imple- 
mented the intervention in the classroom. The 
training consisted of coaching and immediate 
corrective feedback. Training continued until 
teachers implemented the plan with 100% 
integrity on two consecutive occasions with 
the consultant’s assistance. Teachers were pro- 
vided all necessary materials (e.g., tokens or 
sticker charts, reinforcers, or protocols) for 
implementation in this and subsequent phases. 

Implementation baseline. Following initial 
training, teachers implemented the plan with- 
out assistance or feedback from the consultant. 
Trained observers collected data on integrity of 
plan implementation by the teacher and on 
students’ target behaviors. 

Goal setting and student performance feedback. 
The purpose of this phase was to examine the 
accuracy of teachers’ plan implementation after 


setting a goal for student behavior and then 
receiving feedback about the student’s progress 
toward that goal. Students’ goals were de- 
termined based on their pretraining baseline 
levels and were set at a 50% reduction from this 
baseline mean. Once treatment integrity de- 
creased and was stable (two consecutive data 
points within 20 percentage points) following 
initial training, the consultant met with each 
teacher to set a goal for student behavior based 
on baseline performance. During this phase, 
teachers were provided with daily written 
feedback and time-series line graphs of student 
behavior. Although each teacher received daily 
feedback regarding his or her student’s behav- 
ior, he or she did not receive feedback regarding 
the accuracy of implementation of the in- 
tervention. Once treatment integrity decreased 
or was stable, the next phase was introduced. 

Teacher performance feedback and directed 
rehearsal with meeting cancellation. The purpose 
of this phase was to examine the effects of 
performance feedback combined with negative 
reinforcement in the form of avoiding a meeting 
to practice missed steps (i.e., directed rehearsal). 
This phase was introduced when treatment 
integrity decreased from the previous phase and 
was stable. Teachers were provided with daily 
written feedback and time-series line graphs of 
their performance and that of their respective 
students. If a teacher did not obtain 100% 
integrity on that day’s observation, a meeting 
with the consultant was held the following day 
prior to the next scheduled observation. During 
this meeting, any missed or incorrect steps were 
reviewed and practiced three times. If, however, 
a teacher obtained 100% integrity, the meeting 
with the consultant was not held. At the start of 
this phase, the consultant explicitly described 
the contingency to each teacher. A performance 
criterion of three consecutive days with 100% 
integrity was required before moving to the next 
phase. 

Fading. All procedures from the previous 
condition were in place; however, teachers 
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received reduced performance feedback if in- 
tegrity remained high. Specifically, if integrity 
was maintained at 100% for three consecutive 
observations, the schedule was thinned to once 
every other day, once per week and, sub- 
sequently, once every 2 weeks. Teachers who 
did not maintain 100% integrity for three 
consecutive observations were returned to the 
previous schedule until the criterion was again 
met. Due to time constraints associated with the 
school calendar, the schedules were not thinned 
to once every 2 weeks for all teachers. 

RESULTS 
Treatment Integrity 

The percentage of treatment steps imple- 
mented by the teachers and the percentage of 
intervals of student target behavior across all 
phases of the study are presented in Figures 1 
and 2. During pretraining baseline, teachers 
taught their classes as they normally would and 
responded to severe problem behavior as out- 
lined in each student’s behavioral intervention 
plan that was in place before the start of this 
study. Because the individualized interventions 
implemented for the purposes of this investiga- 
tion were not in place yet, teachers demonstrat- 
ed zero percentages of treatment integrity. 
Teachers were then instructed in how to use 
the intervention and were provided with the 
consultant’s assistance in the training phase. 
Teachers A, C, and D met the training criterion 
(implementation at 100% for two consecutive 
sessions) in four or fewer sessions, whereas 
Teacher B required 15 sessions to reach 
criterion due to high variability in implementa- 
tion integrity. 

Once each teacher met the training criterion, 
the consultant discontinued her assistance. 
Specifically, the consultant no longer provided 
cues to implement a treatment step and did not 
provide immediate corrective feedback. The 
data in Figures 1 and 2 show an immediate 
drop in intervention implementation by all 4 
teachers. Teachers A, B, and C showed little to 


no use of the intervention throughout the 
implementation baseline. By the end of this 
phase. Teacher D’s implementation level de- 
creased to 46%. 

Once teachers’ implementation decreased 
and was stable, either goal setting and student 
performance feedback or teacher performance 
feedback and directed rehearsal with meeting 
cancellation was introduced. In goal setting and 
student performance feedback, teachers set 
a goal for student behavior with the consultant 
and received daily written feedback and graphs 
of student progress. Teacher A showed an 
increase in treatment integrity levels (M — 
58%) compared to implementation baseline, 
but Teacher B showed no improvement (M = 
0%). During teacher performance feedback and 
directed rehearsal with meeting cancellation, 
teachers received daily written feedback and 
graphs of their treatment integrity and of their 
students’ progress. Putative negative reinforce- 
ment was contingent on accurate implementa- 
tion of the intervention in the form of avoiding 
a directed rehearsal meeting with the consultant 
to review every missed or incorrectly imple- 
mented step. Teacher A demonstrated the most 
rapid improvement in treatment integrity and 
averaged 100% accuracy. Teacher B was 
somewhat slower to demonstrate consistent 
improvements in treatment integrity, but did 
so within five sessions {M — 79%). 

During teacher performance feedback and 
directed rehearsal with meeting cancellation, 
Teacher C showed slower increases in treatment 
integrity (M = 74%) than did Teacher D (M = 
97%); however, both met performance criterion 
and were accurately implementing the class- 
room intervention within five sessions. During 
goal setting and student performance feedback, 
Teacher C exhibited an increase in treatment 
integrity (M = 67%) compared to implementa- 
tion baseline levels of 0%. Teacher D showed 
similar treatment integrity in implementation 
baseline (M — 66%) and goal setting and 
student performance feedback (M — 63%). 
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Figure 1. Percentage of treatment steps implemented by Teachers A and B and the percentage of intervals of target 
problem behavior by Students A and B across all phases of the study. 


Following goal setting and student performance 
feedback, a brief reversal to teacher performance 
feedback and directed rehearsal with meeting 
cancellation occurred for Teachers C and D. 
Both teachers demonstrated higher treatment 
integrity percentages during these conditions 
than during goal setting and student perfor- 
mance feedback. Teacher C averaged 100% 
accuracy and Teacher D averaged 97% accuracy 


during the second implementation of teacher 
performance feedback and directed rehearsal 
with meeting cancellation. Thus, wi thin-subject 
functional control over teacher performance 
feedback and directed rehearsal with meeting 
cancellation condition was demonstrated with 
Teachers C and D. 

When the performance feedback schedule 
was thinned in the fading phase, all teachers 
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Figure 2. Percentage of treatment steps implemented by Teachers C and D and the percentage of intervals of target 
problem behavior by Students C and D across all phases of the study. 


continued to demonstrate high levels of treat- 
ment integrity. The reinforcement schedule was 
thinned to every other day for Teacher C, once 
per week for Teachers B and D, and once every 
2 weeks for Teacher A. Teacher A averaged 
98% accurate implementation. Teacher B 
averaged 91%, Teacher C averaged 98%, and 
Teacher D averaged 97%. 


Treatment Effectiveness 

Although the primary focus of this investi- 
gation was on teacher treatment integrity, 
student behavior was also measured as a second- 
ary dependent variable. In pretraining baseline, 
students’ target behaviors were high and showed 
substantial variability. On average, Students A, 
B, and D displayed off-task behavior in more 
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than half of the intervals (59%, 91%, and 77%, 
respectively). Student C engaged in work refusal 
an average of 79% of intervals. 

All students exhibited reductions in target 
behaviors upon introduction of the interven- 
tion. Students A, B, and D displayed off-task 
behavior an average of 24%, 49%, and 18% of 
intervals, respectively, during the training phase. 
Student C also showed decreases in work refusal 
(M = 45%) compared to levels in pretraining 
baseline. 

With the introduction of goal setting and 
student performance feedback, only 2 of the 4 
students showed substantial reductions in their 
target behaviors (Students B and C). Student A 
showed a slight reduction in off-task behavior 
(M = 56%), and Student B’s off-task behavior 
averaged 26% of intervals. The percentage of 
intervals during which Student C displayed 
work refusal decreased to an average of 37% 
during this condition. Student D’s off-task 
behavior declined slightly to a mean of 68%. 

Compared to pretraining baseline levels, all 
students exhibited reductions in problem be- 
havior during teacher performance feedback 
and directed rehearsal with meeting cancella- 
tion. Three of the 4 students showed systematic 
decreases in their target behaviors based on 
visual inspection of the data. In this condition, 
off-task behavior averaged 30%, 50%, and 20% 
for Students A, B, and D, respectively. Student 
C engaged in work refusal an average of 38% of 
intervals. Although these percentages are lower 
than pretraining baseline levels, Student C did 
not show a substantial difference in work refusal 
during teacher performance feedback and di- 
rected rehearsal with meeting cancellation 
compared to goal setting and student perfor- 
mance feedback. In addition, Student B 
demonstrated higher levels of off-task behavior 
in the former condition. 

Decreases in problem behavior were main- 
tained for 3 of the 4 students during fading. 
Students A, B, C, and D averaged 41%, 50%, 
41%, and 11%, respectively. 


Correlational Analyses 

The relation between teacher integrity and 
student target behavior was calculated to de- 
termine the extent to which increased accuracy 
of implementation was associated with inter- 
vention effectiveness. The two variables were 
significantly correlated for Dyad A, r — — .45, p 
< .05; for Dyad C, r = —.66 , p < .01; and for 
Dyad D, r — —.78, p < .01. A statistically 
significant correlation was not found for Dyad 
B, r = .02. 

Treatment Acceptability 

Teachers’ responses on the IRP-15 were 
obtained to determine the acceptability of the 
intervention. Total acceptability scores ranged 
from 59 to 88 (M — 78) of 90, indicating 
general intervention acceptability. The mean 
item rating across all teachers was 5.2 (of 6), 
with 3 of the 4 teachers slightly agreeing to 
strongly agreeing with each item (ratings of 4, 5, 
or 6). Teacher B slightly disagreed with 33% of 
the items (a rating of 3). These items pertained 
to effectiveness and benefit of the intervention, 
suitability and fairness of the intervention given 
the student’s problem behavior, and the extent 
to which he or she liked the procedures used in 
the intervention. 

Teacher responses on the modified IRP 
provided information specific to the use of the 
integrity interventions (goal setting and student 
performance feedback or teacher performance 
feedback and directed rehearsal with meeting 
cancellation). Teachers indicated a general ac- 
ceptability of the use of goal setting and student 
performance feedback, with total acceptability 
scores ranging from 49 to 60 [M — 57) of 60. 
The mean item rating across all teachers was 5.7 
(of 6), with all teachers slightly agreeing to 
strongly agreeing with each item. For example, 
all 4 teachers agreed or strongly agreed that 
setting goals for student behavior and receiving 
daily written feedback about student perfor- 
mance were acceptable ways to help teachers 
accurately implement plans for students. Teach- 
ers indicated a general acceptability of teacher 
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performance feedback and directed rehearsal 
with meeting cancellation, with total accept- 
ability scores ranging from 69 to 1 07 (M — 87) 
of 108. The mean item rating across all teachers 
was 4.8 (of 6), with 2 of the 4 teachers slightly 
agreeing to strongly agreeing with each item. All 
teachers agreed or strongly agreed on items 
specifying that the procedures used would be 
beneficial for teachers and that they liked the 
procedures used to assist them in implementing 
the intervention. However, Teachers A and B 
expressed slight disagreement or disagreement 
with statements pertaining to practicing missed 
intervention steps and avoiding a meeting. For 
example, Teacher A disagreed (a rating of 2) 
with the statements that avoiding a meeting or 
practicing missed intervention steps was an 
acceptable way to help teachers implement 
plans for students. Similarly, Teacher B in- 
dicated slight disagreement with the statement 
that she would be willing to practice missed 
intervention steps again in the future. The 
component of teacher performance feedback 
and directed rehearsal with meeting cancellation 
that Teachers A and B found most acceptable 
was the receipt of daily written feedback. On 
average, teachers rated both procedures as 
acceptable, with goal setting and student 
performance feedback rated as slightly more 
acceptable. 

DISCUSSION 

The goal of this study was to examine the 
extent to which treatment integrity of 4 special 
education classroom teachers was affected by 
two different strategies (goal setting and student 
performance feedback; teacher performance 
feedback and directed rehearsal with meeting 
cancellation). The results replicate and extend 
previous findings suggesting that a teacher 
performance feedback package plus directed 
rehearsal with meeting cancellation can effec- 
tively increase teachers’ treatment integrity in 
special education classrooms (DiGennaro et ah, 
2005). In addition, high treatment integrity was 


associated with lower percentages of student 
problem behavior for three of the four dyads. 
This relation also replicates findings by DiGen- 
naro et al. 

This study provides some evidence that daily 
meetings may not be necessary to maintain 
treatment integrity over time. Instead, high 
levels of integrity were demonstrated when 
teachers were allowed to avoid a meeting with 
a consultant that included directed rehearsal 
following written performance feedback. These 
results replicate the findings of DiGennaro et al. 
(2005) showing that accurate plan implementa- 
tion can be maintained on leaner schedules. The 
present findings suggest that implicit positive 
reinforcement contingencies (i.e., receiving 
written feedback about a student’s progress 
toward a preestablished goal) may not be 
sufficient to maintain teacher treatment in- 
tegrity. This is unfortunate, given that the goal- 
setting procedures were generally time-efficient 
for both the consultant and teacher and that 
teachers rated these procedures as more accept- 
able. 

Despite receiving initial training superior to 
that typically provided to teachers when a new 
plan is introduced, implementation integrity 
decreased substantially when consultant assis- 
tance was removed. This finding emphasizes the 
importance of providing teachers with ongoing 
support, specifically feedback about their own 
performance. Inclusion of directed rehearsal 
permits focused, repeated practice of missed 
components of an intervention plan in the 
needed areas. Thus, allowing teachers to 
practice a skill and then avoid meeting with 
a consultant once skill acquisition in the natural 
setting is observed appears to be an effective 
means to promote treatment integrity. As stated 
previously, a critical aspect of effective consul- 
tation is to ensure that teachers have acquired 
the skills needed for plan implementation and 
that teacher behavior change has occurred 
(Erchul & Martens, 2002). The present find- 
ings suggest that use of performance feedback, 
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directed rehearsal, and meeting cancellation as 
a packaged intervention can produce desired 
changes in teacher behavior. Finally, the negative 
correlation found between teachers’ implementa- 
tion integrity and students’ problem behavior for 
3 of the 4 teachers underscores the importance of 
consultants monitoring and programming for 
treatment integrity in their work with teachers. 
These findings have direct implications for 
behavioral consultants. This study demonstrated 
an effective application of behavioral consultation 
that incorporated applied behavior-analytic tech- 
niques (e.g., Martens, Witt, Daly, & Vollmer, 
1999). Components of this consultation process 
included (a) functional assessments to inform the 
design of school-based interventions; (b) model- 
ing, coaching, and prompting to train teachers to 
implement interventions; (c) antecedent- and 
consequence-based interventions for teachers and 
students; (d) formative evaluation of teachers and 
students; and (e) data-based decision making. 

There were several limitations that should be 
noted and may be addressed in future research. 
First, it is unknown what components of 
teacher performance feedback and directed 
rehearsal with meeting cancellation may have 
produced increases in treatment integrity. For 
example, either directed rehearsal of missed 
steps or performance feedback about teacher 
behavior may have been sufficient to strengthen 
teachers’ repertoires enough to promote high 
levels of implementation integrity. On the other 
hand, the meeting cancellation contingency 
alone or in combination with these other 
practice variables may have accounted for the 
results that were obtained. Future research 
should examine the effects of these interventions 
implemented in isolation before combining 
them into a packaged intervention. However, 
it would be impossible to examine either 
directed rehearsal or meeting cancellation in 
the absence of some type of feedback. 

One can presume that the teachers contacted 
negative reinforcement through the meeting 
cancellation contingency or directed rehearsal. 


We do not know, however, if the consultant 
will take on aversive properties or if unintended 
long-term consequences will result if this is 
indeed the case. Considering the teacher survey 
results of Costenbader, Swartz, and Petrix 
(1992), who found that the most frequently 
reported barrier to effective school-based con- 
sultation was lack of teacher time, we believe 
that the teachers in this study may also have 
been motivated to engage in behaviors that 
saved time (Elliot, 1988) and were not simply 
avoiding the consultant. Furthermore, only 2 of 
the teachers rated the procedures used in teacher 
performance feedback and directed rehearsal 
with meeting cancellation as unacceptable. 
Thus, teachers’ implementation behavior may 
have been strengthened through a positive 
reinforcement contingency (i.e., more time to 
engage in preferred activities) or through escape 
or avoidance of unpleasant consequences (i.e., 
practicing missed steps). Additional investiga- 
tions are warranted to examine this issue more 
fully and to examine what effects, if any, use of 
directed rehearsal with meeting cancellation has 
on the consultation relationship. 

Third, Teacher B consistently demonstrated 
variability in her treatment integrity, suggesting 
a limitation of the strategies used to facilitate 
teacher performance. During the initial train- 
ing, she did not always respond effectively to 
prompting by the consultant even though she 
originally agreed to the treatment plan. In 
addition, her accuracy in implementation 
remained variable through much of teacher 
performance feedback and directed rehearsal 
with meeting cancellation. This is probably 
related to the relatively low acceptability ratings 
she gave Student B’s intervention and the 
directed rehearsal with meeting cancellation 
components, and suggests that ongoing assess- 
ment of the social acceptability of both student 
and teacher interventions may be an important 
component for effective classroom consultation. 

Next, student performance remained variable 
throughout all phases of the study. This may 
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have been due to our reliance on indirect and 
descriptive functional assessments to design 
interventions appropriate for each dyad. It is 
possible that different interventions may have 
been developed had we conducted a functional 
analysis; having teachers implement a different 
set of interventions, in turn, would have likely 
influenced the correlation between treatment 
integrity and student problem behavior. 

Finally, an additional limitation is that the 
support schedules for 3 of the 4 teachers were 
not thinned to the desired goal (i.e., feedback 
provided every 2 weeks). This is an important 
area for future research. 

Notwithstanding these limitations, the 
study’s main findings suggest that teacher 
performance feedback with a meeting avoidance 
contingency can be an effective means to 
promote teacher treatment integrity in special 
education classrooms. Furthermore, integrity 
levels were higher under these conditions than 
when teachers were provided feedback on the 
extent to which the students were meeting 
agreed-upon behavioral goals. Finally, this study 
demonstrated that improvements in student 
behavior are indeed associated with implemen- 
tation accuracy, thus providing perhaps the 
most compelling reason to continue the search 
for strategies that promote accurate treatment 
implementation in the absence of a consultant. 
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